Normalize plugin secret/connection refs + add usages SDK surface#521
Open
RhysSullivan wants to merge 6 commits intomainfrom
Open
Normalize plugin secret/connection refs + add usages SDK surface#521RhysSullivan wants to merge 6 commits intomainfrom
RhysSullivan wants to merge 6 commits intomainfrom
Conversation
Owner
Author
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
This was referenced May 4, 2026
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| 🔵 In progress View logs |
executor-cloud | 6437df1 | May 05 2026, 06:49 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| 🔵 In progress View logs |
executor-marketing | 6437df1 | May 05 2026, 06:49 PM |
@executor-js/cli
@executor-js/config
@executor-js/execution
@executor-js/sdk
@executor-js/storage-core
@executor-js/codemode-core
@executor-js/runtime-quickjs
@executor-js/plugin-file-secrets
@executor-js/plugin-google-discovery
@executor-js/plugin-graphql
@executor-js/plugin-keychain
@executor-js/plugin-mcp
@executor-js/plugin-onepassword
@executor-js/plugin-openapi
executor
commit: |
RhysSullivan
force-pushed
the
rs/normalize-mcp
branch
from
eca66cf to
45caef2
Compare
RhysSullivan
force-pushed
the
rs/normalize-google-discovery
branch
from
f2d49d3 to
41b0ace
Compare
Plugins gain optional `usagesForSecret` / `usagesForConnection` callbacks that the executor fans out across via `executor.secrets.usages(id)` and `executor.connections.usages(id)`. Secret/connection removal is now RESTRICT — refuses with `SecretInUseError` / `ConnectionInUseError` when any plugin reports the id as in use, surfacing the count to the caller. API gains GET `/secrets/:id/usages` and `/connections/:id/usages` endpoints. React atoms and a small "Used by …" footer on each row in the Secrets and Connections tabs round out the surface. No plugin implements `usagesForSecret`/`usagesForConnection` yet, so the footer renders nothing and remove still succeeds for everything. Plugin migration lands in follow-up commits.
Move secret/connection references out of the JSON columns on graphql_source (auth, headers, query_params) into: - two new flat columns auth_kind + auth_connection_id (indexed) - two child tables graphql_source_header / graphql_source_query_param, each with kind/text_value/secret_id/secret_prefix and an indexed secret_id This lets usagesForSecret and usagesForConnection answer "where is X referenced?" with a single indexed SELECT instead of scanning JSON, and secrets.remove / connections.remove now correctly RESTRICT when a graphql source still references the id. Migration 0007 backfills existing JSON via json_each + json_extract, then drops the old columns. Three tests cover the backfill against a hand-seeded pre-migration DB; three plugin-level tests cover usage fan-out and remove RESTRICT.
Move secret/connection references out of the JSON columns on
openapi_source and openapi_source_binding into:
- openapi_source_binding gains kind/secret_id/connection_id/text_value
columns (replacing the discriminated-union json value)
- openapi_source_query_param: child table for query_params, mirrors
graphql_source_query_param shape
- openapi_source_spec_fetch_header / spec_fetch_query_param: child
tables for invocation_config.specFetchCredentials.{headers,queryParams}
Drops openapi_source.query_params and openapi_source.invocation_config
JSON columns. headers and oauth2 stay JSON because they hold slot
names rather than direct refs — the slots resolve through bindings,
which ARE normalized.
Plugin gains usagesForSecret / usagesForConnection that fan out across
all four locations in one indexed SELECT each. secrets.remove now
correctly RESTRICTs when an openapi binding still uses the secret.
Migration 0008 backfills via json_extract / json_each, then drops the
old columns. Three migration tests cover the backfill from a
hand-seeded pre-migration DB; two plugin-level tests cover usage
fan-out and remove RESTRICT.
Move secret/connection references out of the JSON `mcp_source.config` column into: - Flat columns on mcp_source: auth_kind, auth_header_name, auth_secret_id, auth_secret_prefix, auth_connection_id, auth_client_id_secret_id, auth_client_secret_secret_id (each indexed where it carries a ref). Replaces the McpConnectionAuth json blob. - mcp_source_header / mcp_source_query_param: child tables for the remote source's SecretBackedMap entries (same shape as graphql / openapi child tables). The remaining structural fields (transport, endpoint, command, args, etc.) stay as JSON in `config` because they're plugin-private and vary by transport. mcp_binding.binding stays JSON too — McpToolBinding carries no refs and inputSchema/outputSchema are arbitrary user JSON. Plugin gains usagesForSecret / usagesForConnection. Migration 0009 backfills via json_extract / json_each, then strips the extracted fields with json_remove. Three migration tests cover backfill from a hand-seeded pre-migration DB; two plugin-level tests cover usage fan-out across header-auth and oauth2 sources.
Move secret/connection references out of the JSON google_discovery_source.config column into: - Flat columns on google_discovery_source: auth_kind, auth_connection_id, auth_client_id_secret_id, auth_client_secret_secret_id, auth_scopes (each indexed where it carries a ref). Replaces the GoogleDiscoveryAuth json. - google_discovery_source_credential_header / google_discovery_source_credential_query_param: child tables for the optional GoogleDiscoveryFetchCredentials SecretBackedMap entries. The remaining structural fields (discoveryUrl, service, version, rootUrl, servicePath) stay in `config` as JSON — they're plugin-private and never carry refs. google_discovery_binding stays JSON too: the GoogleDiscoveryMethodBinding has no refs, and method input/output schemas are arbitrary user data. Plugin gains usagesForSecret / usagesForConnection. Migration 0010 backfills via json_extract / json_each, then strips the extracted fields with json_remove. Three migration tests cover backfill from a hand-seeded pre-migration DB; one plugin-level test covers usage fan-out across auth_*_secret_id and auth_connection_id.
RhysSullivan
force-pushed
the
graphite-base/521
branch
from
45caef2 to
5d47fb8
Compare
RhysSullivan
force-pushed
the
rs/normalize-google-discovery
branch
from
41b0ace to
2698722
Compare
RhysSullivan
changed the title
Merge graphql/openapi/mcp/google-discovery normalization into one migration per app: 0007_normalize_plugin_secret_refs (local sqlite) and 0008_normalize_plugin_secret_refs (cloud pg, mcp-only since cloud doesn't ship google-discovery). Each test now imports a shared PRE_0007_SQL helper that seeds all four plugins' parent tables at the pre-migration shape — needed because the single consolidated migration touches every plugin in one transaction.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Plugins (graphql, openapi, mcp, google-discovery) now store every direct secret/connection reference in proper relational columns/child tables instead of JSON blobs. The SDK gains
usagesForSecret/usagesForConnectionso the Secrets and Connections tabs can show what's referencing each one — and so deletes can be RESTRICTed when in use (HTTP 409SecretInUseError/ConnectionInUseError) instead of leaving dangling references.What's in the diff
graphql_source→ flatauth_kind/auth_connection_idcolumns; newgraphql_source_header,graphql_source_query_paramchild tablesopenapi_source_binding→ flatkind/secret_id/connection_id/text_value; newopenapi_source_query_param,openapi_source_spec_fetch_header,openapi_source_spec_fetch_query_parammcp_source→ 7 flat auth columns; newmcp_source_header,mcp_source_query_paramgoogle_discovery_source→ flat auth + scopes columns; newgoogle_discovery_source_credential_header,google_discovery_source_credential_query_paramapps/local(sqlite, 0007–0010) andapps/cloud(pg, 0008–0010). Each backfills viajson_each/jsonb_eachthen drops the old JSON columns.usagesForSecret(scopeId, secretId)/usagesForConnection(scopeId, connectionId)onPluginSpec, fanned out by the executor;SecretInUseError/ConnectionInUseErroron the secrets/connections delete paths;GET /scopes/:scopeId/{secrets,connections}/:id/usagesHTTP endpoints.apps/cloud/src/services/executor-schema.ts.JSON.stringify([sourceId, name])instead of${sourceId}:${name}so user-provided separators can't collide.@rhyssul/portless0.13.0 → 0.13.3.Test plan